Perspectives on the role of industry in informatics research and authorship.

OBJECTIVES: Advances in informatics research come from academic, nonprofit, and for-profit industry organizations, and from academic-industry partnerships. While scientific studies of commercial products may offer critical lessons for the field, manuscripts authored by industry scientists are sometimes categorically rejected. We review historical context, community perceptions, and guidelines on informatics authorship. PROCESS: We convened an expert panel at the American Medical Informatics Association 2022 Annual Symposium to explore the role of industry in informatics research and authorship with community input. The panel summarized session themes and prepared recommendations. CONCLUSIONS: Authorship for informatics research, regardless of affiliation, should be determined by International Committee of Medical Journal Editors uniform requirements for authorship. All authors meeting criteria should be included, and categorical rejection based on author affiliation is unethical. Informatics research should be evaluated based on its scientific rigor; all sources of bias and conflicts of interest should be addressed through disclosure and, when possible, methodological mitigation.

PillHarmonics™: An Orchestrated Pharmacogenetics Medication Clinical Decision Support Service.

OBJECTIVES: Pharmacogenetics (PGx) is increasingly important in individualizing therapeutic management plans, but is often implemented apart from other types of medication clinical decision support (CDS). The lack of integration of pharmacogenetics into existing CDS may result in incomplete interaction information, which may pose patient safety concerns. We sought to develop a cloud-based orchestrated medication CDS service that integrates PGx with a broad set of drug screening alerts and evaluate it through a clinician utility study. METHODS: We developed the PillHarmonicsTM service for implementation per the CDS Hooks protocol, algorithmically integrating a wide range of drug interaction knowledge using cloud-based screening services from First Databank (drug-drug/allergy/condition), PharmGKB (drug-gene), and locally curated content (drug-renal/hepatic/race). We performed a user study, presenting 13 clinicians and pharmacists with a prototype of the system's usage in synthetic patient scenarios. We collected feedback via a standard questionnaire and structured interview. RESULTS: Clinician assessment of PillHarmonics via the Technology Acceptance Model questionnaire shows significant evidence of perceived utility. Thematic analysis of structured interviews revealed that aggregated knowledge, concise actionable summaries, and information accessibility were highly valued, and that clinicians would use the service in their practice. CONCLUSIONS: Medication safety and optimizing efficacy of therapy regimens remain significant issues. A comprehensive medication CDS system that leverages patient clinical and genomic data to perform a wide range of interaction checking and present a concise and holistic view of medication knowledge back to the clinician, is feasible and perceived as highly valuable for more informed decision-making. Such a system can potentially address many of the challenges identified with current medication related CDS.

Implementation and evaluation of an electronic health record-integrated app for postpartum monitoring of hypertensive disorders of pregnancy using patient-contributed data collection.

Remote monitoring of women experiencing hypertensive disorders of pregnancy (HDP) can provide timely life-saving data, particularly if these data are integrated into existing patient and clinical workflows. This pilot intervention of a smartphone application (app) for postpartum monitoring of hypertensive disorders integrates patient-contributed data into electronic health records (EHRs) to support monitoring and clinical decision-making. Results from the evaluation of the pilot highlight the resources needed when implementing the app, challenges for integrating an app into the EHR, and the usability and utility of the HDP monitoring app for patient and clinician users. The implementation team's key observations included the importance of a local clinical champion, more robust patient involvement and support for the remote patient monitoring program, an impetus for EHR developers to adopt data integration standards, and a need to expand the capabilities of the standards to support interventions using patient-contributed data.

Sync for Genes Phase 5: Computable artifacts for sharing dynamically annotated FHIR-formatted genomic variants.

Introduction: Variant annotation is a critical component in next-generation sequencing, enabling a sequencing lab to comb through a sea of variants in order to hone in on those likely to be most significant, and providing clinicians with necessary context for decision-making. But with the rapid evolution of genomics knowledge, reported annotations can quickly become out-of-date. Under the ONC Sync for Genes program, our team sought to standardize the sharing of dynamically annotated variants (e.g., variants annotated on demand, based on current knowledge). The computable biomedical knowledge artifacts that were developed enable a clinical decision support (CDS) application to surface up-to-date annotations to clinicians. Methods: The work reported in this article relies on the Health Level 7 Fast Healthcare Interoperability Resources (FHIR) Genomics and Global Alliance for Genomics and Health (GA4GH) Variant Annotation (VA) standards. We developed a CDS pipeline that dynamically annotates patient's variants through an intersection with current knowledge and serves up the FHIR-encoded variants and annotations (diagnostic and therapeutic implications, molecular consequences, population allele frequencies) via FHIR Genomics Operations. ClinVar, CIViC, and PharmGKB were used as knowledge sources, encoded as per the GA4GH VA specification. Results: Primary public artifacts from this project include a GitHub repository with all source code, a Swagger interface that allows anyone to visualize and interact with the code using only a web browser, and a backend database where all (synthetic and anonymized) patient data and knowledge are housed. Conclusions: We found that variant annotation varies in complexity based on the variant type, and that various bioinformatics strategies can greatly improve automated annotation fidelity. More importantly, we demonstrated the feasibility of an ecosystem where genomic knowledge bases have standardized knowledge (e.g., based on the GA4GH VA spec), and CDS applications can dynamically leverage that knowledge to provide real-time decision support, based on current knowledge, to clinicians at the point of care.

Visualization of Patient-Generated Health Data: A Scoping Review of Dashboard Designs.

BACKGROUND: Patient-centered clinical decision support (PC CDS) aims to assist with tailoring decisions to an individual patient's needs. Patient-generated health data (PGHD), including physiologic measurements captured frequently by automated devices, provide important information for PC CDS. The volume and availability of such PGHD is increasing, but how PGHD should be presented to clinicians to best aid decision-making is unclear. OBJECTIVES: Identify best practices in visualizations of physiologic PGHD, for designing a software application as a PC CDS tool. METHODS: We performed a scoping review of studies of PGHD dashboards that involved clinician users in design or evaluations. We included only studies that used physiologic PGHD from single patients for usage in decision-making. RESULTS: We screened 468 titles and abstracts, 63 full-text papers, and identified 15 articles to include in our review. Some research primarily sought user input on PGHD presentation; other studies garnered feedback only as a side effort for other objectives (e.g., integration with electronic health records [EHRs]). Development efforts were often in the domains of chronic diseases and collected a mix of physiologic parameters (e.g., blood pressure and heart rate) and activity data. Users' preferences were for data to be presented with statistical summaries and clinical interpretations, alongside other non-PGHD data. Recurrent themes indicated that users desire longitudinal data display, aggregation of multiple data types on the same screen, actionability, and customization. Speed, simplicity, and availability of data for other purposes (e.g., documentation) were key to dashboard adoption. Evaluations were favorable for visualizations using common graphing or table formats, although best practices for implementation have not yet been established. CONCLUSION: Although the literature identified common themes on data display, measures, and usability, more research is needed as PGHD usage grows. Ensuring that care is tailored to individual needs will be important in future development of clinical decision support.

A lifecycle framework illustrates eight stages necessary for realizing the benefits of patient-centered clinical decision support.

The design, development, implementation, use, and evaluation of high-quality, patient-centered clinical decision support (PC CDS) is necessary if we are to achieve the quintuple aim in healthcare. We developed a PC CDS lifecycle framework to promote a common understanding and language for communication among researchers, patients, clinicians, and policymakers. The framework puts the patient, and/or their caregiver at the center and illustrates how they are involved in all the following stages: Computable Clinical Knowledge, Patient-specific Inference, Information Delivery, Clinical Decision, Patient Behaviors, Health Outcomes, Aggregate Data, and patient-centered outcomes research (PCOR) Evidence. Using this idealized framework reminds key stakeholders that developing, deploying, and evaluating PC-CDS is a complex, sociotechnical challenge that requires consideration of all 8 stages. In addition, we need to ensure that patients, their caregivers, and the clinicians caring for them are explicitly involved at each stage to help us achieve the quintuple aim.

Introducing HL7 FHIR Genomics Operations: a developer-friendly approach to genomics-EHR integration.

OBJECTIVE: Enabling clinicians to formulate individualized clinical management strategies from the sea of molecular data remains a fundamentally important but daunting task. Here, we describe efforts towards a new paradigm in genomics-electronic health record (HER) integration, using a standardized suite of FHIR Genomics Operations that encapsulates the complexity of molecular data so that precision medicine solution developers can focus on building applications. MATERIALS AND METHODS: FHIR Genomics Operations essentially "wrap" a genomics data repository, presenting a uniform interface to applications. More importantly, operations encapsulate the complexity of data within a repository and normalize redundant data representations-particularly relevant in genomics, where a tremendous amount of raw data exists in often-complex non-FHIR formats. RESULTS: Fifteen FHIR Genomics Operations have been developed, designed to support a wide range of clinical scenarios, such as variant discovery; clinical trial matching; hereditary condition and pharmacogenomic screening; and variant reanalysis. Operations are being matured through the HL7 balloting process, connectathons, pilots, and the HL7 FHIR Accelerator program. DISCUSSION: Next-generation sequencing can identify thousands to millions of variants, whose clinical significance can change over time as our knowledge evolves. To manage such a large volume of dynamic and complex data, new models of genomics-EHR integration are needed. Qualitative observations to date suggest that freeing application developers from the need to understand the nuances of genomic data, and instead base applications on standardized APIs can not only accelerate integration but also dramatically expand the applications of Omic data in driving precision care at scale for all.

Integrating a Patient Engagement App into an Electronic Health Record-Enabled Workflow Using Interoperability Standards.

BACKGROUND: Patient use of mobile health applications is increasing. To promote patient-centered care, data from these apps must be integrated into clinician workflows within the electronic health record (EHR). Health Level 7 Fast Healthcare Interoperability Resources (FHIR) offers a standards-based application programming interface (API) that may support such integration. OBJECTIVE: We aimed to use interoperability standards to integrate a patient mobile application (coronavirus 2019 [COVID-19] Tracker) with an EHR. The COVID-19 Tracker engages patients by sending introductory and reminder text messages, collecting vital signs and symptom data from COVID-19 patients, and providing actionable guidance if concerning issues are identified. This case report explored the use of FHIR APIs to integrate the app into EHR-enabled clinical workflows. METHODS: The authors used notes from project meetings and from semistructured discussions among the application development team to track the design and implementation processes. Seven points of integration between the application and the EHR were identified, and approaches using FHIR to perform these integrations were delineated. RESULTS: Although this clinical decision support integration project benefited from its standards-based approach, many challenges were encountered. These were due to (1) partial implementation of the FHIR standard in the EHR, particularly, components needed for patient engagement applications; (2) limited experience with the adoption of FHIR standards; and (3) gaps in the current FHIR standard. Alternative approaches, often not based on interoperability standards, were developed to overcome these limitations. CONCLUSION: Despite the challenges encountered due to the early stages of FHIR development and adoption, FHIR standards provide a promising mechanism for overcoming longstanding barriers and facilitating the integration of patient engagement apps with EHRs. To accelerate the integration of apps into clinical workflows, additional components of the FHIR standard must be implemented within the EHR and other clinical systems. Continued expansion of available FHIR resources will help with tighter workflow integration.

The Technology Landscape of Patient-Centered Clinical Decision Support - Where Are We and What Is Needed?

Patient Centered Outcomes Research (PCOR) and health care delivery system transformation require investments in development of tools and techniques for rapid dissemination of clinical and operational best practices. This paper explores the current technology landscape for patient-centered clinical decision support (PC CDS) and what is needed to make it more shareable, standards-based, and publicly available with the goal of improving patient care and clinical outcomes. The landscape assessment used three sources of information: (1) a 22-member technical expert panel; (2) a literature review of peer-reviewed and grey literature; and (3) key informant interviews with PC CDS stakeholders. We identified ten salient technical considerations that span all phases of PC CDS development; our findings suggest there has been significant progress in the development and implementation of PC CDS but challenges remain.

Challenges and opportunities for advancing patient-centered clinical decision support: findings from a horizon scan.

OBJECTIVE: We conducted a horizon scan to (1) identify challenges in patient-centered clinical decision support (PC CDS) and (2) identify future directions for PC CDS. MATERIALS AND METHODS: We engaged a technical expert panel, conducted a scoping literature review, and interviewed key informants. We qualitatively analyzed literature and interview transcripts, mapping findings to the 4 phases for translating evidence into PC CDS interventions (Prioritizing, Authoring, Implementing, and Measuring) and to external factors. RESULTS: We identified 12 challenges for PC CDS development. Lack of patient input was identified as a critical challenge. The key informants noted that patient input is critical to prioritizing topics for PC CDS and to ensuring that CDS aligns with patients' routine behaviors. Lack of patient-centered terminology standards was viewed as a challenge in authoring PC CDS. We found a dearth of CDS studies that measured clinical outcomes, creating significant gaps in our understanding of PC CDS' impact. Across all phases of CDS development, there is a lack of patient and provider trust and limited attention to patients' and providers' concerns. DISCUSSION: These challenges suggest opportunities for advancing PC CDS. There are opportunities to develop industry-wide practices and standards to increase transparency, standardize terminologies, and incorporate patient input. There is also opportunity to engage patients throughout the PC CDS research process to ensure that outcome measures are relevant to their needs. CONCLUSION: Addressing these challenges and embracing these opportunities will help realize the promise of PC CDS-placing patients at the center of the healthcare system.

Usability of a perioperative medication-related clinical decision support software application: a randomized controlled trial.

OBJECTIVE: We developed a comprehensive, medication-related clinical decision support (CDS) software prototype for use in the operating room. The purpose of this study was to compare the usability of the CDS software to the current standard electronic health record (EHR) medication administration and documentation workflow. MATERIALS AND METHODS: The primary outcome was the time taken to complete all simulation tasks. Secondary outcomes were the total number of mouse clicks and the total distance traveled on the screen in pixels. Forty participants were randomized and assigned to complete 7 simulation tasks in 1 of 2 groups: (1) the CDS group (n = 20), who completed tasks using the CDS and (2) the Control group (n = 20), who completed tasks using the standard medication workflow with retrospective manual documentation in our anesthesia information management system. Blinding was not possible. We video- and audio-recorded the participants to capture quantitative data (time on task, mouse clicks, and pixels traveled on the screen) and qualitative data (think-aloud verbalization). RESULTS: The CDS group mean total task time (402.2 ± 85.9 s) was less than the Control group (509.8 ± 103.6 s), with a mean difference of 107.6 s (95% confidence interval [CI], 60.5-179.5 s, P < .001). The CDS group used fewer mouse clicks (26.4 ± 4.5 clicks) than the Control group (56.0 ± 15.0 clicks) with a mean difference of 29.6 clicks (95% CI, 23.2-37.6, P < .001). The CDS group had fewer pixels traveled on the computer monitor (59.5 ± 20.0 thousand pixels) than the Control group (109.3 ± 40.8 thousand pixels) with a mean difference of 49.8 thousand pixels (95% CI, 33.0-73.7, P < .001). CONCLUSIONS: The perioperative medication-related CDS software prototype substantially outperformed standard EHR workflow by decreasing task time and improving efficiency and quality of care in a simulation setting.

The technical landscape for patient-centered CDS: progress, gaps, and challenges.

Supporting healthcare decision-making that is patient-centered and evidence-based requires investments in the development of tools and techniques for dissemination of patient-centered outcomes research findings via methods such as clinical decision support (CDS). This article explores the technical landscape for patient-centered CDS (PC CDS) and the gaps in making PC CDS more shareable, standards-based, and publicly available, with the goal of improving patient care and clinical outcomes. This landscape assessment used: (1) a technical expert panel; (2) a literature review; and (3) interviews with 18 CDS stakeholders. We identified 7 salient technical considerations that span 5 phases of PC CDS development. While progress has been made in the technical landscape, the field must advance standards for translating clinical guidelines into PC CDS, the standardization of CDS insertion points into the clinical workflow, and processes to capture, standardize, and integrate patient-generated health data.

Development of a Perioperative Medication-Related Clinical Decision Support Tool to Prevent Medication Errors: An Analysis of User Feedback.

OBJECTIVES: Medication use in the perioperative setting presents many patient safety challenges that may be improved with electronic clinical decision support (CDS). The objective of this paper is to describe the development and analysis of user feedback for a robust, real-time medication-related CDS application designed to provide patient-specific dosing information and alerts to warn of medication errors in the operating room (OR). METHODS: We designed a novel perioperative medication-related CDS application in four phases: (1) identification of need, (2) alert algorithm development, (3) system design, and (4) user interface design. We conducted group and individual design feedback sessions with front-line clinician leaders and subject matter experts to gather feedback about user requirements for alert content and system usability. Participants were clinicians who provide anesthesia (attending anesthesiologists, nurse anesthetists, and house staff), OR pharmacists, and nurses. RESULTS: We performed two group and eight individual design feedback sessions, with a total of 35 participants. We identified 20 feedback themes, corresponding to 19 system changes. Key requirements for user acceptance were: Use hard stops only when necessary; provide as much information as feasible about the rationale behind alerts and patient/clinical context; and allow users to edit fields such as units, time, and baseline values (e.g., baseline blood pressure). CONCLUSION: We incorporated user-centered design principles to build a perioperative medication-related CDS application that uses real-time patient data to provide patient-specific dosing information and alerts. Emphasis on early user involvement to elicit user requirements, workflow considerations, and preferences during application development can result in time and money efficiencies and a safer and more usable system.

vcf2fhir: a utility to convert VCF files into HL7 FHIR format for genomics-EHR integration.

BACKGROUND: VCF formatted files are the lingua franca of next-generation sequencing, whereas HL7 FHIR is emerging as a standard language for electronic health record interoperability. A growing number of FHIR-based clinical genomics applications are emerging. Here, we describe an open source utility for converting variants from VCF format into HL7 FHIR format. RESULTS: vcf2fhir converts VCF variants into a FHIR Genomics Diagnostic Report. Conversion translates each VCF row into a corresponding FHIR-formatted variant in the generated report. In scope are simple variants (SNVs, MNVs, Indels), along with zygosity and phase relationships, for autosomes, sex chromosomes, and mitochondrial DNA. Input parameters include VCF file and genome build ('GRCh37' or 'GRCh38'); and optionally a conversion region that indicates the region(s) to convert, a studied region that lists genomic regions studied by the lab, and a non-callable region that lists studied regions deemed uncallable by the lab. Conversion can be limited to a subset of VCF by supplying genomic coordinates of the conversion region(s). If studied and non-callable regions are also supplied, the output FHIR report will include 'region-studied' observations that detail which portions of the conversion region were studied, and of those studied regions, which portions were deemed uncallable. We illustrate the vcf2fhir utility via two case studies. The first, 'SMART Cancer Navigator', is a web application that offers clinical decision support by linking patient EHR information to cancerous gene variants. The second, 'Precision Genomics Integration Platform', intersects a patient's FHIR-formatted clinical and genomic data with knowledge bases in order to provide on-demand delivery of contextually relevant genomic findings and recommendations to the EHR. CONCLUSIONS: Experience to date shows that the vcf2fhir utility can be effectively woven into clinically useful genomic-EHR integration pipelines. Additional testing will be a critical step towards the clinical validation of this utility, enabling it to be integrated in a variety of real world data flow scenarios. For now, we propose the use of this utility primarily to accelerate FHIR Genomics understanding and to facilitate experimentation with further integration of genomics data into the EHR.

A secure system for genomics clinical decision support.

We developed a prototype genomic archiving and communications system to securely store genome data and provide clinical decision support (CDS). This system operates on a client-server model. The client encrypts the data, and the server stores data and performs the computations necessary for CDS. Computations are directly performed on encrypted data, and the client decrypts results. The server cannot decrypt inputs or outputs, which provides strong guarantees of security. We have validated our system with three genomics-based CDS applications. The results demonstrate that it is possible to resolve a long-standing dilemma in genomic data privacy and accessibility, by using a principled cryptographical framework and a mathematical representation of genome data and CDS questions.

A system to build distributed multivariate models and manage disparate data sharing policies: implementation in the scalable national network for effectiveness research.

BACKGROUND: Centralized and federated models for sharing data in research networks currently exist. To build multivariate data analysis for centralized networks, transfer of patient-level data to a central computation resource is necessary. The authors implemented distributed multivariate models for federated networks in which patient-level data is kept at each site and data exchange policies are managed in a study-centric manner. OBJECTIVE: The objective was to implement infrastructure that supports the functionality of some existing research networks (e.g., cohort discovery, workflow management, and estimation of multivariate analytic models on centralized data) while adding additional important new features, such as algorithms for distributed iterative multivariate models, a graphical interface for multivariate model specification, synchronous and asynchronous response to network queries, investigator-initiated studies, and study-based control of staff, protocols, and data sharing policies. MATERIALS AND METHODS: Based on the requirements gathered from statisticians, administrators, and investigators from multiple institutions, the authors developed infrastructure and tools to support multisite comparative effectiveness studies using web services for multivariate statistical estimation in the SCANNER federated network. RESULTS: The authors implemented massively parallel (map-reduce) computation methods and a new policy management system to enable each study initiated by network participants to define the ways in which data may be processed, managed, queried, and shared. The authors illustrated the use of these systems among institutions with highly different policies and operating under different state laws. DISCUSSION AND CONCLUSION: Federated research networks need not limit distributed query functionality to count queries, cohort discovery, or independently estimated analytic models. Multivariate analyses can be efficiently and securely conducted without patient-level data transport, allowing institutions with strict local data storage requirements to participate in sophisticated analyses based on federated research networks.

Understanding data requirements of retrospective studies.

BACKGROUND AND OBJECTIVE: Usage of data from electronic health records (EHRs) in clinical research is increasing, but there is little empirical knowledge of the data needed to support multiple types of research these sources support. This study seeks to characterize the types and patterns of data usage from EHRs for clinical research. MATERIALS AND METHODS: We analyzed the data requirements of over 100 retrospective studies by mapping the selection criteria and study variables to data elements of two standard data dictionaries, one from the healthcare domain and the other from the clinical research domain. We also contacted study authors to validate our results. RESULTS: The majority of variables mapped to one or to both of the two dictionaries. Studies used an average of 4.46 (range 1-12) data element types in the selection criteria and 6.44 (range 1-15) in the study variables. The most frequently used items (e.g., procedure, condition, medication) are often available in coded form in EHRs. Study criteria were frequently complex, with 49 of 104 studies involving relationships between data elements and 22 of the studies using aggregate operations for data variables. Author responses supported these findings. DISCUSSION AND CONCLUSION: The high proportion of mapped data elements demonstrates the significant potential for clinical data warehousing to facilitate clinical research. Unmapped data elements illustrate the difficulty in developing a complete data dictionary.

Structured representation for core elements of common clinical decision support interventions to facilitate knowledge sharing.

At present, there are no widely accepted, standard approaches for representing computer-based clinical decision support (CDS) intervention types and their structural components. This study aimed to identify key requirements for the representation of five widely utilized CDS intervention types: alerts and reminders, order sets, infobuttons, documentation templates/forms, and relevant data presentation. An XML schema was proposed for representing these interventions and their core structural elements (e.g., general metadata, applicable clinical scenarios, CDS inputs, CDS outputs, and CDS logic) in a shareable manner. The schema was validated by building CDS artifacts for 22 different interventions, targeted toward guidelines and clinical conditions called for in the 2011 Meaningful Use criteria. Custom style sheets were developed to render the XML files in human-readable form. The CDS knowledge artifacts were shared via a public web portal. Our experience also identifies gaps in existing standards and informs future development of standards for CDS knowledge representation and sharing.

Identifying appropriate reference data models for comparative effectiveness research (CER) studies based on data from clinical information systems.

INTRODUCTION: The need for a common format for electronic exchange of clinical data prompted federal endorsement of applicable standards. However, despite obvious similarities, a consensus standard has not yet been selected in the comparative effectiveness research (CER) community. METHODS: Using qualitative metrics for data retrieval and information loss across a variety of CER topic areas, we compare several existing models from a representative sample of organizations associated with clinical research: the Observational Medical Outcomes Partnership (OMOP), Biomedical Research Integrated Domain Group, the Clinical Data Interchange Standards Consortium, and the US Food and Drug Administration. RESULTS: While the models examined captured a majority of the data elements that are useful for CER studies, data elements related to insurance benefit design and plans were most detailed in OMOP's CDM version 4.0. Standardized vocabularies that facilitate semantic interoperability were included in the OMOP and US Food and Drug Administration Mini-Sentinel data models, but are left to the discretion of the end-user in Biomedical Research Integrated Domain Group and Analysis Data Model, limiting reuse opportunities. Among the challenges we encountered was the need to model data specific to a local setting. This was handled by extending the standard data models. DISCUSSION: We found that the Common Data Model from the OMOP met the broadest complement of CER objectives. Minimal information loss occurred in mapping data from institution-specific data warehouses onto the data models from the standards we assessed. However, to support certain scenarios, we found a need to enhance existing data dictionaries with local, institution-specific information.

A patient-driven adaptive prediction technique to improve personalized risk estimation for clinical decision support.

OBJECTIVE: Competing tools are available online to assess the risk of developing certain conditions of interest, such as cardiovascular disease. While predictive models have been developed and validated on data from cohort studies, little attention has been paid to ensure the reliability of such predictions for individuals, which is critical for care decisions. The goal was to develop a patient-driven adaptive prediction technique to improve personalized risk estimation for clinical decision support. MATERIAL AND METHODS: A data-driven approach was proposed that utilizes individualized confidence intervals (CIs) to select the most 'appropriate' model from a pool of candidates to assess the individual patient's clinical condition. The method does not require access to the training dataset. This approach was compared with other strategies: the BEST model (the ideal model, which can only be achieved by access to data or knowledge of which population is most similar to the individual), CROSS model, and RANDOM model selection. RESULTS: When evaluated on clinical datasets, the approach significantly outperformed the CROSS model selection strategy in terms of discrimination (p<1e-14) and calibration (p<0.006). The method outperformed the RANDOM model selection strategy in terms of discrimination (p<1e-12), but the improvement did not achieve significance for calibration (p=0.1375). LIMITATIONS: The CI may not always offer enough information to rank the reliability of predictions, and this evaluation was done using aggregation. If a particular individual is very different from those represented in a training set of existing models, the CI may be somewhat misleading. CONCLUSION: This approach has the potential to offer more reliable predictions than those offered by other heuristics for disease risk estimation of individual patients.